<html>
<head><meta charset="utf-8"><title>llvm-project aarch64 builds corrupt · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html">llvm-project aarch64 builds corrupt</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="238462353"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238462353" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Gemmell <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238462353">(May 12 2021 at 12:06)</a>:</h4>
<p>Last week I started getting linker errors when using the prebuilt LLVM binaries on an aarch64-unknown-linux-gnu host.</p>
<div class="codehilite"><pre><span></span><code>/usr/local/bin/ld: /tmp/rustcGW4Gt3/librustc_llvm-639595dcc23d4f08.rlib(X86Disassembler.cpp.o): Relocations in generic ELF (EM: 62)
/usr/local/bin/ld: /tmp/rustcGW4Gt3/librustc_llvm-639595dcc23d4f08.rlib(X86Disassembler.cpp.o): error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
</code></pre></div>
<p>The .a lib files are seemingly all for x86_64 (EM: 62 refers to x86_64, which supports this), though the binaries work just fine. Building LLVM locally works fine which suggests that it's an infra problem. Myself and a colleague have reproduced this on other machines.</p>
<p>The only weird thing is that <code>objdump -d</code> on the aarch64 host seems to fail for all the previous artifacts I could find - could have sworn I was using the llvm downloads just fine a few weeks ago but I could have made a mistake of course</p>
<p>cc <span class="user-mention" data-user-id="121055">@Pietro Albini</span></p>



<a name="238463894"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238463894" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238463894">(May 12 2021 at 12:20)</a>:</h4>
<p><span class="user-mention" data-user-id="382356">@Adam Gemmell</span> hmm, I don't have the time to investigate this right now unfortunately</p>



<a name="238463939"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238463939" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238463939">(May 12 2021 at 12:20)</a>:</h4>
<p>how old was the last artifact you found?</p>



<a name="238464878"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238464878" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238464878">(May 12 2021 at 12:28)</a>:</h4>
<p>I think a bisection is the best way to make progress</p>



<a name="238471250"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238471250" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Gemmell <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238471250">(May 12 2021 at 13:16)</a>:</h4>
<p>1st December, from rust-lang/rust <a href="https://github.com/rust-lang/rust/commit/6645da366eed0c61258a04265bea513e94df7ea6">6645da366eed0c61258a04265bea513e94df7ea6</a>. I used this git query <a href="https://github.com/rust-lang/rust/blob/ac923d94f86a6f7c881ecbedcd0a68d7986a35bd/src/bootstrap/bootstrap.py#L468">https://github.com/rust-lang/rust/blob/ac923d94f86a6f7c881ecbedcd0a68d7986a35bd/src/bootstrap/bootstrap.py#L468</a>, and commits it finds after this 404 for aarch64 linux</p>



<a name="238471431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238471431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238471431">(May 12 2021 at 13:17)</a>:</h4>
<p>That is probably the 144 day timeline?</p>



<a name="238471454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238471454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238471454">(May 12 2021 at 13:17)</a>:</h4>
<p>(which is when artifacts entirely vanish)</p>



<a name="238471550"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238471550" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238471550">(May 12 2021 at 13:18)</a>:</h4>
<p>Which sounds like a different problem from the one you described earlier</p>



<a name="238600989"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238600989" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Gemmell <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238600989">(May 13 2021 at 09:42)</a>:</h4>
<blockquote>
<p>I think a bisection is the best way to make progress</p>
</blockquote>
<p>I didn't think I had much to bisect considering all the artifacts are broken and I couldn't reproduce it locally. Last night, however, I ran the dist-aarch64-linux build. It's hosted on x86_64, so it builds LLVM twice - once natively and once as aarch64. The packaged LLVM files in <code>dist/rust-dev-nightly-aarch64-unknown-linux-gnu.tar.xz</code> are all x86, so I think that narrows it down to the packaging scripts which is enough for a bug report. I'll probably come back to fix it later</p>



<a name="238606232"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238606232" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Gemmell <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238606232">(May 13 2021 at 10:49)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/issues/85250">https://github.com/rust-lang/rust/issues/85250</a></p>



<a name="238624153"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238624153" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238624153">(May 13 2021 at 13:55)</a>:</h4>
<p><span class="user-mention" data-user-id="382356">@Adam Gemmell</span> Does that seem like it might be the cause for the build failure in  <a href="https://github.com/rust-lang-ci/rust/runs/2524738098">https://github.com/rust-lang-ci/rust/runs/2524738098</a> as well, or does that seem like an unrelated issue?</p>



<a name="238624314"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238624314" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238624314">(May 13 2021 at 13:57)</a>:</h4>
<p>I'm trying to debug that failure on aarch64-gnu, which is blocking a PR I'm trying to make, and I don't see how my PR would cause it.</p>



<a name="238624393"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238624393" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238624393">(May 13 2021 at 13:57)</a>:</h4>
<p>We don't download LLVM on CI, ever, and that's a native builder, so I wouldn't expect any cross-compilation... but it doesn't rule it out as a possibility, I guess.</p>



<a name="238624812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238624812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238624812">(May 13 2021 at 14:01)</a>:</h4>
<p>It does look like a freshly built LLVM, but yeah, I'm wondering if the issue could potentially occur with a built LLVM.</p>



<a name="238630933"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238630933" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Amanieu <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238630933">(May 13 2021 at 14:44)</a>:</h4>
<p>I hit this issue just now :(</p>



<a name="238632123"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/llvm-project%20aarch64%20builds%20corrupt/near/238632123" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Gemmell <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/llvm-project.20aarch64.20builds.20corrupt.html#238632123">(May 13 2021 at 14:53)</a>:</h4>
<p>I couldn't see any issues with artifacts in <code>obj/build/aarch64-unknown-linux-gnu/llvm</code>, only the final dist .tar.xz files</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>